A Controlled Experiment to Explore Potentially Undetectable Defects for Testing Techniques

Software testing practitioners have an array of testing techniques to choose from to test their software. Nevertheless, there is little empirical evidence about the capability of each technique to detect specific types of defects. As a result, when selecting and combining the testing techniques for a project, practitioners must rely on their own experience. This paper studies the behaviour of two specific techniques, equivalence partitioning and decision coverage, to determine which types of defect are potentially undetectable to either one. This paper presents a differentiated experiment replication based on a previous experimental design, but using different artifacts. The experiment confirms the hypothesis that some defect types are undetectable to each technique. Even with a correct application of each technique, some defects will only be detected by chance. This study adds new empirical evidence for constructing a classification of defects that takes into account technique detection capabilities.